Usability Evaluation Methods The Reliability and Usage of Cognitive Walkthrough and Usability Test

The thesis includes this 98-pages summary and appendices containing a paper submitted for journal publication Contents Our 1 contribution to the field of human-computer interaction (HCI) has been centered on evaluating aspects of two usability evaluation methods (UEMs), namely the cognitive walkthrough and the usability test. Chapter 1 in this summary is devoted to an introduction to HCI with particular focus on UEMs, while chapter 2 contains a description of the two methods investigated (cognitive walk-through and usability test). The most extensive study was the investigation of the learning process and the first time usage of cognitive walkthrough; this work is described in chapter 3. While working on this case we began studying the effect of different evaluators analyzing the same interface with the same UEM, the so-called evaluator effect. We have studied the evaluator effect both in cognitive walkthrough and usability test. In chapter 4 we describe the evaluator effect in cognitive walkthrough. In chapter 5 we describe the evaluator effect in usability test. The last chapter, chapter 6, discusses and draws perspectives on the work described in chapters 3, 4, and 5. Four of the five appendices are included in this thesis submission in order to document earlier reports on the evaluator effect studies. The study about the evaluator effect in cognitive walkthrough has previously been reported in Hertzum & Jacobsen (1998) and Hertzum & Jacobsen (1999), and is included in appendices 2 and 3 respectively. The study about the evaluator effect in usability test has previously been reported in Jacobsen, Hertzum & John (1998a) and Jacobsen, Hertzum & John (1998b), and are included in appendices 4 and 5 respectively. As the early reports on the evaluator effect were constrained with regard to number of pages, I have extended the descriptions in chapters 4 (the evaluator effect in cognitive walkthrough) and 5 (the evaluator effect in usability test) in this summary. Hence, reading these two chapters would be sufficient to understand the main contents of the previous four reports. In addition, chapters 4 and 5 offer some new analyses that have not been revealed in the earlier reports. Opposed to the somewhat repeated reports in appendices 2, 3, 4, and 5, appendix 1 (Jacobsen & John, 1999) contains an extensive description of a case study, which is not repeated in the summary. Hence, chapter 3 introduces learning and usage of cognitive walk-through, but only as a prelude to …